S+: Efficient 2D Sparse LU Factorization on Parallel Machines

Static symbolic factorization coupled with supernode partitioning and asynchronous computation scheduling can achieve high giga op rates for parallel sparse LU factorization with partial pivoting This paper studies properties of elimination forests and uses them to optimize supernode partitioning amalgamation and execution scheduling It also proposes supernodal matrix multiplication to speed up kernel computation by retaining the BLAS level e ciency and avoiding unnecessary arithmetic operations The experiments show that our new design with proper space optimization called S improves our previous solution substantially and can achieve up to GFLOPS on Cray T E MHz nodes Introduction The solution of sparse linear systems is a computational bot tleneck in many scienti c computing problems When dynamic pivoting is required to maintain numerical stability in direct methods for solving non symmetric linear systems it is challenging to develop high performance parallel code because pivoting causes severe caching miss and load imbalance on modern architectures with mem ory hierarchies The previous work has addressed parallelization on shared mem ory platforms or with restricted pivoting Most notably the recent shared memory implementation of SuperLU has achieved up to GFLOPS on Cray C nodes For distributed memory machines we proposed an ap proach that adopts a static symbolic factorization scheme to avoid data structure variation Static symbolic factorization eliminates the runtime overhead of dynamic symbolic factorization with a price of over estimated ll ins and thereafter extra computation However the static data structure allowed us to identify data regularity maximize the use of BLAS operations and utilize task graph scheduling techniques and e cient run time support to achieve high e ciency This paper addresses three issues to further improve the performance of paral lel sparse LU factorization with partial pivoting on distributed memory machines First we study the use of elimination trees in optimizing matrix partitioning and task scheduling Elimination trees or forests are used extensively in sparse Cholesky factorization because they have a more compact representation of paral lelism than task graphs For sparse LU factorization the traditional approach uses the elimination tree of AA which can produce excessive false computational depen dency In this paper we use elimination trees forest of A to guide matrix partitioning and parallelism control in LU factorization We show that improved supernode par titioning and amalgamation e ectively control extra ll ins and produce optimized supernodal partitioning We also use elimination forests to identify data dependence and potential concurrency among pivoting and updating tasks and thus maximize utilization of limited parallelism Second we propose a fast and space e cient kernel for supernode based matrix This work was supported in part by NSF CCR and by DARPA through UMD ONR Contract Number N C yDepartment of Computer Science University of California at Santa Barbara CA USA kshen cs ucsb edu zDepartment of Computer Science University of California at Santa Barbara CA USA tyang cs ucsb edu xDepartment of Computer Science University of Illinois at Urbana Champaign IL USA